Classification and Multiple Regression through Projection Pursuit*

Projection pursuit regression is generalized to multivariate responses. By viewing classification as a special case, this generalization serves to extend classification and discriminant analysis via the projection pursuit approach. Submitted to Journal of the American Statistical Association * Work supported by the Department of Energy under contract DEAC03-76SF00515, by the Office of Naval Research under contract N00014-83-K-0472 and by the U.S. Army Research Office under contract DAAG29-82-K-0056. 1. Multiple Regression Regression is a method for modeling a set of response variables Y; (1 5 i 5 q) as functions of a set of predictor variables Xj (1 5 j <_ p) based on matched observations (training data). ylk!y2k,“‘yqk,=lk,Z2kr”‘2pk (0) Often there is only a single response variable (q = 1). Usually the goal is to estimate the conditional expectation of each Yi given a set of values for the predictor variables I Yi(2lrZ29-*-, Z~)=E[Y~IX~=~~,X~=Z~,“‘,X~=ZP] (l<i<q), (1) as the predictor variable values range over some region of interest in RP. These conditional expectation estimates are then used as best guesses for the true underlying response values assuming that the observed responses were generated from a noisy process yi 7 gi(Xl,X2,“*, xp)+Si (15i<.q) (2) where the 9; are single valued functions of p variables and si is a random variable with zero expectation. The conditional expectations Yi(z,, zz, + . , zp) can be regarded as estimates for the gi(zr,z2,‘+‘,zp) (1 5 i 5 q). The classical linear model expresses the Pi as linear functions of the predictor variables P I Yi(Zl . . * Zp) = Qio + C aij2j j=l where the values of the oij are chosen to be those for which the expected distance between Yi and Yi is minimized. Several different distance measures are in common use, but the most common is the Euclidean A2 (aio ’ * . sip) = l&,X [Yi Pi]“. (3) The resulting estimates are termed least-squares estimates.